On the Relationship between Generalization and Robustness to Adversarial Examples

One of the most intriguing phenomenons related to deep learning is so-called adversarial examples. These samples are visually equivalent normal inputs, undetectable for humans, yet they cause networks output wrong results. The phenomenon can be framed as a symmetry/asymmetry problem, whereby inputs neural network with similar/symmetric appearance regular images, produce an opposite/asymmetric output. Some researchers focused on developing methods generating examples, while others propose defense methods. In parallel, there growing interest in characterizing phenomenon, which also focus this paper. From some well known datasets common like CIFAR-10 and STL-10, architecture first trained regime, where training validation performances increase, reaching generalization. Additionally, same architectures overfitting disparity performances. behaviour these two regimes against examples then compared. results, we observe greater robustness regime. We explain simultaneous loss generalization gain another manifestation well-known fitting-generalization trade-off.